leesedol part 1
===


### General Questions ###



investigator: Okay, recording in progress. Now you should "see general questions regarding usage of digital libraries". 

leesedol: Yes. 


##### GQ1 #####


investigator: Okay. So, let's go to the first question. Which tasks do you usually use a digital library for? Please tick all answers which apply and complete your own tasks. Please give short oral examples of the tasks you're ticking. So, for example, you could say "I tick person search, because I like to keep track of myself". So, which tasks do you usually use a digital library for? 

leesedol: Normally I search for specific terms for a broader context. I never use person search, maybe I search for similar publications, and I would guess that's most of it.

investigator: Okay, so what are you ticking? 

leesedol: "Get BibTeX data", "get full text paper", "study relations". What about "query term search"? 

investigator: Yeah, this would be then one of the tasks that you would include under "Sonstiges".

leesedol: Okay.

investigator: Okay. That's it? 

leesedol: Yeah. 


##### GQ2 #####


investigator: Okay. Then we can continue to the next question. Which system or digital library do you usually use to solve these tasks? Please tick all answers, which apply and name others, which also apply, but are not given here. Please give short oral descriptions while you like, or why use a specific system.

leesedol: Okay. I use Bibsonomy because there are relevant information together with the search term and with the publication. Easy to understand. Also, I use Google scholar because it's easy to use and it's fast and I have very good experience with it. I will... I use research gate, because it's kind of library, which will give you much information. It also offer possibility to see some figures and like that and it brings a good overview of the publication. I also use normal search engine, like Google or something. Because you also get some information which is next of scientific information and public information. So, you can combine these and maybe you find some interesting blogs and so on. So, I would guess that's most. 

investigator: Okay. Is there anything else that's not listed here, but that would you also consider a system or digital library that you use to solve these tasks?

leesedol: Yeah, I would also use arXiv to get the full text and that's okay. It's enough to to type in "arXiv" in this? 

investigator: Yes. Sure. 

leesedol: Okay. That's these are my favorites, I think. 

investigator: Okay, nice. Then I think we can continue to the next page. 

leesedol: Yeah. 


### TASK 1 ###


investigator: Okay. Now you should see task one. Yeah. Consider the following task, find two experts on the topic of your liking. Example topics could be domain specific query languages or hashing functions but should be from a general or broader area of computer and information science. You can pick whatever topic you like. So, it could be anything not just one of the two that are given as examples. And you do not really have to do this task right now.

So, we are only going to talk about how you would solve this task. 

leesedol: Okay. 


##### TASK 1.1 #####


investigator: So, my first question is what is your chosen topic? 

leesedol: "Explainable AI".


##### TASK 1.2 #####


investigator: Great. And how familiar are you with this topic? 

leesedol: I would say out of five, I would say three. 


##### TASK 1.3 #####


investigator: Okay, nice. And how would you define an expert? 

leesedol: An expert is able to define all relevant under topics of explainable AI. He will know about the relevant literature he will know about relevant frameworks to apply explainable AI. He would be able to differentiates between the specific methods, which are publicated in explainable AI and so on. Maybe he had some own publications in this field, which are highly cited, that would be a plus, but not necessary. 

investigator: Okay. Anything else you want to add to this definition? Or is this your full definition. 

leesedol: I think it's full. Yeah. 


##### TASK 1.4 #####


investigator: Okay, perfect. So, then we can continue to question four. How would you solve the task with your topic? So how would you find two experts on a topic of explainable AI? 

leesedol: Two experts? Okay. Maybe I would find the most relevant publications in the field of explainable AI in the last 10 years. And investigate if there are authors who show up more than others. So, I would say these are the most knowledgeable people in this field. Maybe they have publications which are high cited, maybe they show up in many other publications in the introduction field and that would be a strategy to find these experts. 

investigator: Okay. so how would you identify the most relevant papers in this topic? Which system would you use? What would be your starting point? Would you start from a keyword query? would you use a tool? 

leesedol: Normally I would use Google scholar and type in the relevant terms. For my example, it's explainable AI and I will investigate some overview papers, some surveys and investigate how are there any popular or important publications, which show up more often. And I would investigate to them and look if they're good in a scientific way that I would mark these authors as experts. 

investigator: Okay. So how would you find surveys in Google scholar?

leesedol: I would use the query terms, "explainable AI survey", or "explainable AI overview", would use explainable. 

investigator: Okay. And you mentioned that if you find those papers, which you consider relevant or most relevant papers from the last 10 years, you would look at authors of the papers. Which authors do you look at and how do you do that?

Do you start with the first author, or do you have the specific strategy to only look at specific authors and where would you look them up? And yeah. 

leesedol: I would note all authors in the author area and investigate if they are show up more often, if there are many publications where the same author would show up more often, I would consider this as an expert.

investigator: Okay. would you do this also in Google scholar, or would you switch the system, or would you use another system additionally, to check their profile or their other publications? 

leesedol: Normally, I would only use Google scholar if I find the relevant information there. 

investigator: Okay. So, then you mentioned you would check their publications if they're highly cited?

leesedol: Yes. 

investigator: You also do this in Google scholar or? 

leesedol: Yes, I would only use Google scholar for this. 

investigator: Okay. Do you have a specific threshold what you consider highly cited or do you compare them among each other or. 

leesedol: Yes, I would compare them among each other. Because it's been explainable AI is a relatively new field and I cannot have a good threshold on any data cause it's domain specific.

investigator: Okay. And then you also mentioned that you would look at the introduction of papers to see who was mentioned multiple times or often. how do you do that? Do you get the full text of what paper? And where do you get it? 

leesedol: So, I described the strategy to find some relevant papers and I would take these papers and make a qualitative investigation in the introduction. I would read all introductions and I would check if these authors, which are mentioned in the introduction would show up more often than expected. 

investigator: Okay. So, do you get the full text for that? Or do you check the introduction? 

leesedol: Yes, I would get the full text. 

investigator: Okay. How would you get the full text from which webpage or?

leesedol: Okay. If the full text is not available at arXiv or directly as linked as a PDF reference, if it's not public available, maybe I would consider other sources. If these other sources would not make them available I would discard this paper. 

investigator: Okay. So, now that you have like a pool of persons with their citation counts and you know how often they are mentioned in the introductions of relevant papers and what other papers they have written, how do you choose the experts?

leesedol: Okay. If I have a pool of some authors, I would make a qualitative analysis of the paper. If the paper and the writing style, the scientific "niveau" (German word, "level") would correspond to my knowledge. And if I think these papers have good quality, I would, yeah. Consider these people as an expert.

investigator: Okay. 

leesedol: I would not use any more Scientometric data to consider them. 

investigator: Okay. Nice. So, do you feel like this depicts your process fully of finding experts? 

leesedol: Yes. 


### TASK 2 ###


investigator: Okay. Perfect. Then we can continue to the next page with task two. 

Consider the following task find relevant papers from a topic of your liking, which appeared after 2017. Example topics could be paper recommendation or author disambiguation but should also be from a broader area of computer and information science. You can also pick the same topic as before. So, you can also pick explainable AI. 


##### TASK 2.1 #####


First question. What is your chosen topic? 

leesedol: "Explainable AI". 


##### TASK 2.2 #####
##### TASK 2.3 #####


investigator: Okay, perfect. So, we can skip question two of your familiarity with the topic and go to question three, which asks you, how would you define relevancy?

leesedol: Okay. A relevant paper should consider other publications, which are publicated in the last years, which achieved good performance on relevant tasks. So, I would consider relevant papers, which builds up in a scientific way on all the publications. 

investigator: Okay. 

leesedol: Also, relevant papers should be understandable and scientifically correct. And and in the best case they should also publicate data and some source codes. 

investigator: Okay. Anything else you want to add to that? Or should we continue with the next question?

leesedol: Let me think about it. Yeah, that's good. Okay. That's everything. 


##### TASK 2.4 #####


investigator: Okay. Perfect. So, then how would you solve this task? How would you find relevant papers from the area of explainable AI, which appeared after 2017? I think almost all of them appeared after 2017, but how would you find relevant papers? 

leesedol: Okay. I would use the query term explainable AI. At Google scholar you are able to filter the year. So, I would only filter for publications after 2017 and in the next step, I would specify my search terms to have a subset of publications, which are more useful to me. For example, I would specify explainable AI in relation extraction. So, I would add query terms to specify my search. 

investigator: Okay. How do you find those added query terms or how do you define them? 

leesedol: These are the additional query terms I would derive them from my specific research. So, I'm interested in relation extraction in explainable AI. So, I add these query terms. Maybe if I don't understand something, in explainable AI, I would also use a query term, which I don't understand, but which is domain specific in explainable AI.

And I would choose this query term, for example, global methods in explainable AI. If I don't understand that I would add this query term to my search. 

investigator: Okay. Do you get the not understood query terms from the titles of papers that appeared from your last search or, ...?

leesedol: No. In the beginning I would search for overview papers. I'm completely new in the field and ...

investigator: No, no, no. Consider your current familiarity with the field. So, you can consider it like with your knowledge right now. 

leesedol: Okay. If there is no other paper referenced in my specific paper, which I'm using currently where there's something that I don't understand, I would, the first step, I would try to find a paper, which is referenced in the paper at first. And if it's not I would go back to Google scholar and search for these query terms, which are in that paper. 

investigator: Okay. And now that you have all these different papers resulting your searches, how do you choose the ones that are relevant or yeah. Or which ones are relevant from these? 

leesedol: I think the, the question is really general. It depends on my current task. If it's only if I'm completely new in the field, I would say relevant papers are papers which have high quality and low processing effort. So, I will find these more relevant, but if I'm more familiar, I would choose papers, which are more specific to my current research, which are more near to my publications. 

investigator: Okay. How do you check for the quality of the paper? You mentioned that if you are not that familiar with the topic you would search for high quality papers. 

leesedol: Yes. I think I would check if there's any conflict between my scientific knowledge between these papers. I would say the quality is not so good. Also if there are any figures or good descriptions of any scientific ideas, I would say the quality is better. So, if the processing effort is not too much, if there are referenced other good quality papers, I would say the quality is better. If they came from a good conference or good journals, I would say the quality is better if there are popular names, which I identified as good authors, I would say the quality is better. 

investigator: Okay. How do you know which venue is good or do you just know the venues from this area already? Or how would you check if the venue was good? 

leesedol: I don't have a specific measure. It's all from experience. It's like a trust network. If there are other people who say this conference is good, or this conference is bad, and I have specific experience, maybe I visited this venue and had bad experiences so that some of specific experience and trust to other researchers. 

investigator: Okay. So, you would talk to others to find out?

leesedol: Yes. 

investigator: Okay, perfect. yeah. So, you would get the full text of these papers and check them to determine their quality and if they're relevant, is that correct?

leesedol: Yes. 

investigator: Okay. And the full text you get from where? 

leesedol: Normally from google scholar or arXiv or research gate. I think these are the main sources. 

investigator: Okay, perfect. So, do you want to add anything else to this question? Or do you think this depicts your process fully? 

leesedol: Yes. 


### Thank you ###


investigator: Okay. Then I can stop the recording.

